OpenStack Liberty : Use Cinder Storage(Multi-BackEnds)
2015/12/28 |
It's possible to use Virtual Storages provided by Cinder
if an Instance needs more disks.
Configure Virtual storage with NFS and GlusterFS Multi-Backend on here.
+------------------+ +------------------+ 10.0.0.50| [ Storage Node ] | 10.0.0.61| | +------------------+ +-----+ Cinder-Volume | +-----+ GlusterFS #1 | | [ Control Node ] | | eth0| | | eth0| | | Keystone |10.0.0.30 | +------------------+ | +------------------+ | Glance |------------+------------------------------+ | Nova API |eth0 | +------------------+ | +------------------+ | Cinder API | | eth0| [ Compute Node ] | | eth0| | +------------------+ +-----+ Nova Compute | +-----+ GlusterFS #2 | 10.0.0.51| | 10.0.0.62| | +------------------+ | +------------------+ | | +------------------+ | eth0| | +-----+ NFS | 10.0.0.40| | +------------------+ |
[1] |
NFS server is required to be running on your LAN, refer to here.
On this example, configure "/storage" directory on "nfs.srv.world" as a shared directory. |
[2] |
GlusterFS server is required to be running on your LAN, refer to here.
This example uses a replication volume "vol_replica" provided by "glfs01" and "glfs02". |
[3] | Storage ノードã®è¨å®šã§ã™ã€‚ |
root@storage:~#
apt-get -y install nfs-common glusterfs-client
root@storage:~#
vi /etc/cinder/cinder.conf # add follows in the [DEFAULT] section enabled_backends = nfs,glusterfs # add follows to the end
[nfs]
volume_driver = cinder.volume.drivers.nfs.NfsDriver volume_backend_name = NFS nfs_shares_config = /etc/cinder/nfs_shares nfs_mount_point_base = $state_path/mnt_nfs
[glusterfs]
volume_driver = cinder.volume.drivers.glusterfs.GlusterfsDriver volume_backend_name = GlusterFS glusterfs_shares_config = /etc/cinder/glusterfs_shares glusterfs_mount_point_base = $state_path/mnt_glusterfs
root@storage:~#
vi /etc/cinder/nfs_shares # create new : specify NFS shared directories nfs.srv.world:/storage
root@storage:~#
vi /etc/cinder/glusterfs_shares # create new : specify GlusterFS volumes glfs01.srv.world:/vol_replica chmod 640 /etc/cinder/nfs_shares /etc/cinder/glusterfs_shares root@storage:~# chgrp cinder /etc/cinder/nfs_shares /etc/cinder/glusterfs_shares root@storage:~# initctl restart cinder-volume |
[4] | Configure Compute Node to mount NFS and GlusterFS volumes. |
root@node01:~#
apt-get -y install nfs-common glusterfs-client
root@node01:~#
vi /etc/nova/nova.conf # add follows in the [DEFAULT] section osapi_volume_listen = 0.0.0.0 volume_api_class = nova.volume.cinder.API initctl restart nova-compute |
[5] | Create volume types. It's OK to work on any node. (This example is on Control Node) |
# set environment variable first root@dlp ~(keystone)# echo "export OS_VOLUME_API_VERSION=2" >> ~/keystonerc root@dlp ~(keystone)# source ~/keystonerc
cinder type-create nfs +--------------------------------------+------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+------+-------------+-----------+ | b153f4c3-1e73-4fca-bd01-47d95a4f20ea | nfs | - | True | +--------------------------------------+------+-------------+-----------+root@dlp ~(keystone)# cinder type-create glusterfs +--------------------------------------+-----------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+-----------+-------------+-----------+ | 98a035f4-7570-4c70-8a1c-b56b6f05249e | glusterfs | - | True | +--------------------------------------+-----------+-------------+-----------+root@dlp ~(keystone)# cinder type-list +--------------------------------------+-----------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+-----------+-------------+-----------+ | 98a035f4-7570-4c70-8a1c-b56b6f05249e | glusterfs | - | True | | b153f4c3-1e73-4fca-bd01-47d95a4f20ea | nfs | - | True | +--------------------------------------+-----------+-------------+-----------+ |
[6] | Associates volume types set in [5] and names set in cinder.conf. |
root@dlp ~(keystone)# cinder type-key nfs set volume_backend_name=NFS root@dlp ~(keystone)# cinder type-key glusterfs set volume_backend_name=GlusterFS root@dlp ~(keystone)# cinder extra-specs-list +--------------------------------------+-----------+----------------------------------------+ | ID | Name | extra_specs | +--------------------------------------+-----------+----------------------------------------+ | 98a035f4-7570-4c70-8a1c-b56b6f05249e | glusterfs | {u'volume_backend_name': u'GlusterFS'} | | b153f4c3-1e73-4fca-bd01-47d95a4f20ea | nfs | {u'volume_backend_name': u'NFS'} | +--------------------------------------+-----------+----------------------------------------+ |
[7] | Create volumes with specifying volume types. |
root@dlp ~(keystone)# cinder create --display_name disk_nfs --volume-type nfs 10 +---------------------------------------+--------------------------------------+ | Property | Value | +---------------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2016-01-02T08:45:14.000000 | | description | None | | encrypted | False | | id | 32937b19-12f7-4bb3-8743-07e4cd0c94f7 | | metadata | {} | | migration_status | None | | multiattach | False | | name | disk_nfs | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | fa7d34414b8c45c4a2fdc29403fb5cd2 | | os-volume-replication:driver_data | None | | os-volume-replication:extended_status | None | | replication_status | disabled | | size | 10 | | snapshot_id | None | | source_volid | None | | status | creating | | user_id | 5efb3edfd2f347eea6258068a06cb9f1 | | volume_type | nfs | +---------------------------------------+--------------------------------------+root@dlp ~(keystone)# cinder create --display_name disk_glusterfs --volume-type glusterfs 10 +---------------------------------------+--------------------------------------+ | Property | Value | +---------------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2016-01-02T08:45:19.000000 | | description | None | | encrypted | False | | id | 6581e8eb-ec46-47a7-a162-68d77fe748fc | | metadata | {} | | migration_status | None | | multiattach | False | | name | disk_glusterfs | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | fa7d34414b8c45c4a2fdc29403fb5cd2 | | os-volume-replication:driver_data | None | | os-volume-replication:extended_status | None | | replication_status | disabled | | size | 10 | | snapshot_id | None | | source_volid | None | | status | creating | | user_id | 5efb3edfd2f347eea6258068a06cb9f1 | | volume_type | glusterfs | +---------------------------------------+--------------------------------------+root@dlp ~(keystone)# cinder list +--------------------------------------+-----------+------------------+----------------+------+-------------+----------+-------------+-------------+ | ID | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to | +--------------------------------------+-----------+------------------+----------------+------+-------------+----------+-------------+-------------+ | 32937b19-12f7-4bb3-8743-07e4cd0c94f7 | available | - | disk_nfs | 10 | nfs | false | False | | | 6581e8eb-ec46-47a7-a162-68d77fe748fc | available | - | disk_glusterfs | 10 | glusterfs | false | False | | +--------------------------------------+-----------+------------------+----------------+------+-------------+----------+-------------+-------------+ |
[8] | The Configuration is all OK. Attach volumes to instances like follows. |
root@dlp ~(keystone)# nova list +-----------+-------------+---------+------------+-------------+-----------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +-----------+-------------+---------+------------+-------------+-----------------------------------+ | a25ba5e7- | Ubuntu_1404 | SHUTOFF | - | Shutdown | int_net=192.168.100.3, 10.0.0.201 | +-----------+-------------+---------+------------+-------------+-----------------------------------+root@dlp ~(keystone)# nova volume-attach Ubuntu_1404 32937b19-12f7-4bb3-8743-07e4cd0c94f7 auto +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | 32937b19-12f7-4bb3-8743-07e4cd0c94f7 | | serverId | a25ba5e7-d01d-45ab-8484-f61315fa6648 | | volumeId | 32937b19-12f7-4bb3-8743-07e4cd0c94f7 | +----------+--------------------------------------+root@dlp ~(keystone)# nova volume-attach Ubuntu_1404 6581e8eb-ec46-47a7-a162-68d77fe748fc auto +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdc | | id | 6581e8eb-ec46-47a7-a162-68d77fe748fc | | serverId | a25ba5e7-d01d-45ab-8484-f61315fa6648 | | volumeId | 6581e8eb-ec46-47a7-a162-68d77fe748fc | +----------+--------------------------------------+ # the status of attached disk turns "in-use" like follows root@dlp ~(keystone)# cinder list +-----------+--------+------------------+----------------+------+-------------+----------+-------------+----------------+ | ID | Status | Migration Status | Name | Size | Volume Type | Bootable | Multiattach | Attached to | +-----------+--------+------------------+----------------+------+-------------+----------+-------------+----------------+ | 32937b19- | in-use | - | disk_nfs | 10 | nfs | false | False | a25ba5e7-d01d- | | 6581e8eb- | in-use | - | disk_glusterfs | 10 | glusterfs | false | False | a25ba5e7-d01d- | +-----------+--------+------------------+----------------+------+-------------+----------+-------------+----------------+ |